Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Backport PR #2586 to release/v1.7 for modify rust package structure #2590

Conversation

vdaas-ci
Copy link
Collaborator

@vdaas-ci vdaas-ci commented Aug 31, 2024

Description

SSIA

Related Issue

Versions

  • Vald Version: v1.7.13
  • Go Version: v1.22.6
  • Rust Version: v1.80.0
  • Docker Version: v27.1.1
  • Kubernetes Version: v1.30.3
  • Helm Version: v3.15.3
  • NGT Version: v2.2.4
  • Faiss Version: v1.8.0

Checklist

Special notes for your reviewer

Summary by CodeRabbit

  • New Features

    • Introduced a new Rust package named "algorithm" with an addition function for basic arithmetic operations.
    • Added a new Rust package named "faiss" with initial configuration for future development.
    • Introduced a new Rust package named "ngt" with updated dependencies.
    • Added a new Rust package named "observability" to enhance monitoring capabilities.
  • Bug Fixes

    • Upgraded dependencies in various packages to improve performance, security, and compatibility.
  • Chores

    • Restructured the project workspace to include new libraries, enhancing functionality and modularity.

* modify rust package structure

Signed-off-by: Kosuke Morimoto <ksk@vdaas.org>

* format

Signed-off-by: Kosuke Morimoto <ksk@vdaas.org>

---------

Signed-off-by: Kosuke Morimoto <ksk@vdaas.org>
Co-authored-by: Yusuke Kato <kpango@vdaas.org>
Copy link

cloudflare-workers-and-pages bot commented Aug 31, 2024

Deploying vald with  Cloudflare Pages  Cloudflare Pages

Latest commit: 713fe52
Status: ✅  Deploy successful!
Preview URL: https://3996045e.vald.pages.dev
Branch Preview URL: https://backport-release-v1-7-featur-q5i7.vald.pages.dev

View logs

@vdaas-ci
Copy link
Collaborator Author

[WARNING:CONFLICT] You may require to fix the conflict. Please check.

@vdaas-ci
Copy link
Collaborator Author

[CHATOPS:HELP] ChatOps commands.

  • 🙆‍♀️ /approve - approve
  • 🍱 /format - format codes and add licenses
  • /gen-test - generate test codes
  • 🏷️ /label - add labels
  • 🔚 2️⃣ 🔚 /label actions/e2e-deploy - run E2E deploy & integration test

Copy link
Contributor

coderabbitai bot commented Aug 31, 2024

Important

Review skipped

Review was skipped due to path filters

Files ignored due to path filters (1)
  • rust/Cargo.lock is excluded by !**/*.lock

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Walkthrough

Walkthrough

The changes involve a restructuring of a Rust project's workspace configuration, with the addition of new libraries and updates to existing dependencies across multiple modules. The project now includes libraries for observability and various algorithms, while dependencies have been upgraded to newer versions. The project structure has been modified to reflect these changes, including renaming and reorganizing components.

Changes

File(s) Change Summary
rust/Cargo.toml Updated workspace members to include new libraries: observability, algorithm, ngt, faiss.
rust/bin/agent/Cargo.toml Replaced ngt dependency with algorithm; updated versions for prost, tokio, and tonic.
rust/libs/algorithm/Cargo.toml Renamed package from ngt to algorithm; removed ngt-rs dependency, added faiss and ngt.
rust/libs/algorithm/src/lib.rs Changed add function parameters and return type from usize to u64.
rust/libs/algorithms/faiss/Cargo.toml Introduced new Cargo.toml for faiss package with basic metadata.
rust/libs/algorithms/faiss/src/lib.rs Added add function for unsigned 64-bit integer addition and a test module.
rust/libs/algorithms/ngt/Cargo.toml Renamed package from ngt-rs to ngt; updated dependencies for anyhow, cxx, miette.
rust/libs/algorithms/ngt/src/input.cpp Updated include paths for header files reflecting new directory structure.
rust/libs/algorithms/ngt/src/lib.rs Updated include path for header file in ffi module.
rust/libs/observability/Cargo.toml Introduced new Cargo.toml for observability with specified dependencies.
rust/libs/proto/Cargo.toml Updated versions for prost, tonic, and tonic-types.

Thank you for using CodeRabbit. We offer it for free to the OSS community and would appreciate your support in helping us grow. If you find it useful, would you consider giving us a shout-out on your favorite social media?

Share
Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>.
    • Generate unit testing code for this file.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai generate unit testing code for this file.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai generate interesting stats about this repository and render them as a table.
    • @coderabbitai show all the console.log statements in this repository.
    • @coderabbitai read src/utils.ts and generate unit testing code.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

Outside diff range, codebase verification and nitpick comments (1)
rust/bin/agent/Cargo.toml (1)

24-24: Incomplete replacement of ngt with algorithm.

The transition from ngt to algorithm is not fully executed. The ngt library is still referenced in several files, including Cargo.lock and rust/libs/algorithms/ngt/Cargo.toml. Ensure that all references to ngt are updated to algorithm and remove ngt from the dependency graph if it is no longer needed.

  • rust/Cargo.toml
  • rust/Cargo.lock
  • rust/libs/algorithm/Cargo.toml
  • rust/libs/algorithms/ngt/Cargo.toml
Analysis chain

Review the replacement of ngt with algorithm.

The replacement of ngt with algorithm suggests a significant shift in the underlying library or functionality. This change should be carefully reviewed to ensure that it aligns with the project's goals and does not introduce any regressions or compatibility issues. It's crucial to verify that all references to ngt in the project have been updated to algorithm and that the new library meets the project's requirements.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify all references to `ngt` have been replaced with `algorithm`.

# Test: Search for the old dependency usage. Expect: No occurrences of `ngt`.
rg --type toml -A 5 $'ngt'

Length of output: 1490

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between 7fe3c9c and f2d4fdf.

Files ignored due to path filters (1)
  • rust/Cargo.lock is excluded by !**/*.lock
Files selected for processing (11)
  • rust/Cargo.toml (1 hunks)
  • rust/bin/agent/Cargo.toml (1 hunks)
  • rust/libs/algorithm/Cargo.toml (1 hunks)
  • rust/libs/algorithm/src/lib.rs (1 hunks)
  • rust/libs/algorithms/faiss/Cargo.toml (1 hunks)
  • rust/libs/algorithms/faiss/src/lib.rs (1 hunks)
  • rust/libs/algorithms/ngt/Cargo.toml (1 hunks)
  • rust/libs/algorithms/ngt/src/input.cpp (1 hunks)
  • rust/libs/algorithms/ngt/src/lib.rs (1 hunks)
  • rust/libs/observability/Cargo.toml (1 hunks)
  • rust/libs/proto/Cargo.toml (1 hunks)
Files skipped from review due to trivial changes (4)
  • rust/libs/algorithms/faiss/Cargo.toml
  • rust/libs/algorithms/ngt/src/input.cpp
  • rust/libs/algorithms/ngt/src/lib.rs
  • rust/libs/observability/Cargo.toml
Additional comments not posted (12)
rust/libs/algorithm/Cargo.toml (1)

22-23: Dependency Changes: Introduction of "faiss" and "ngt".

The dependencies have been updated to include "faiss" and "ngt" under a new directory structure. This change supports the broadened scope of the library. It's crucial to verify that these paths are correct and that the dependencies are properly integrated into the build system.

rust/libs/algorithm/src/lib.rs (1)

16-16: Function Signature Change: "add" function now uses u64.

The update from usize to u64 for both parameters and the return type of the add function allows handling of larger integer values. This is a significant change that could affect how the function is used across the project, especially in contexts where the size of integers is critical. Ensure that all calls to this function are updated to pass u64 values.

Verification successful

Function Signature Change Verified: "add" function uses u64.

The test cases for the add function are compatible with the new u64 signature, using integer literals that fit within this type. No other usages of the function were found in the non-test code, indicating that the change does not affect other parts of the codebase. Ensure that any future usage of this function aligns with the u64 type.

Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify the usage of the `add` function across the project.

# Test: Search for incorrect usage of the `add` function.
rg --type rust -A 5 $'add\(' | grep -v 'u64'

Length of output: 905


Script:

#!/bin/bash
# Description: Check the types of arguments passed to the `add` function across the codebase.

# Test: Search for all instances of the `add` function and inspect the argument types.
rg --type rust -A 2 'add\('

Length of output: 714

rust/libs/proto/Cargo.toml (1)

25-27: Dependency Version Updates: "prost", "tonic", and "tonic-types".

The versions of prost, tonic, and tonic-types have been updated. These updates are likely to include improvements and bug fixes. It is crucial to verify that these new versions are compatible with the rest of the project and do not introduce any breaking changes.

rust/libs/algorithms/ngt/Cargo.toml (4)

17-17: Package Name Updated

The package name has been changed from ngt-rs to ngt. This simplification aligns with typical Rust naming conventions and should be reflected across all documentation and import statements in the project.


26-27: Build Dependency Version Updated: cxx-build and miette

The cxx-build dependency has been updated to 1.0.126, and miette has been updated to 7.2.0. These updates should enhance the build process and error handling capabilities. Verify that these new versions integrate smoothly with the build system.

Verification successful

Build Dependency Integration Verified: cxx-build and miette

The updated versions of cxx-build and miette are correctly integrated into the build system. They are actively used in rust/libs/algorithms/ngt/build.rs for building and error handling, respectively. The Cargo.lock file confirms the version updates. No issues found with the integration.

  • rust/libs/algorithms/ngt/build.rs: Utilizes cxx-build and miette.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify integration of new build dependency versions with the build system.

# Test: Search for build scripts using these dependencies. Expect: Smooth integration.
rg --type shell -A 5 $'cxx-build|miette'

Length of output: 71


Script:

#!/bin/bash
# Description: Verify integration of new build dependency versions with the build system.

# Corrected Test: Search for usage of `cxx-build` and `miette` in the codebase.
rg 'cxx-build|miette' -A 5

Length of output: 2079


23-23: Dependency Version Updated: cxx

The cxx dependency version has been updated from 1.0.117 to 1.0.126, maintaining the c++20 feature. This update should be checked for compatibility with the project's C++ components.


22-22: Dependency Version Updated: anyhow

The version of anyhow has been updated from 1.0.81 to 1.0.86. This update likely includes bug fixes and improvements. Ensure that this version is compatible with other dependencies and does not introduce breaking changes.

rust/libs/algorithms/faiss/src/lib.rs (1)

16-18: New Function: Basic Addition

The function add correctly implements a basic addition operation for u64 integers. This is a straightforward implementation and serves as a good example of simple Rust functions.

rust/Cargo.toml (1)

20-27: Updated Workspace Members List

The workspace configuration has been updated to include new libraries and modules (libs/observability, libs/algorithm, libs/algorithms/ngt, libs/algorithms/faiss). This change likely aims to improve project organization and dependency management. Ensure that all new paths are correctly set up and accessible in the project's build system.

Verification successful

Paths Correctly Set Up in Rust Workspace

The new paths (libs/observability, libs/algorithm, libs/algorithms/ngt, libs/algorithms/faiss) are correctly included in the Cargo.toml files, indicating proper setup in the Rust workspace. The algorithm library is also referenced in rust/bin/agent/Cargo.toml, confirming its use as a dependency. No additional build scripts were found referencing these paths, suggesting reliance on standard Rust workspace behavior.

  • rust/Cargo.toml: Lists all new paths as workspace members.
  • rust/bin/agent/Cargo.toml: References algorithm as a dependency.
Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Verify that new paths are correctly set up in the build system.

# Test: Search for references to new paths in build scripts. Expect: Correct setup.
rg --type shell -A 5 $'libs/observability|libs/algorithm|libs/algorithms/ngt|libs/algorithms/faiss'

Length of output: 130


Script:

#!/bin/bash
# Description: Verify that new paths are correctly set up in the build system.

# Test: Search for references to new paths in common build-related files.
rg 'libs/observability|libs/algorithm|libs/algorithms/ngt|libs/algorithms/faiss' --glob '*Makefile' --glob '*build.rs' --glob '*.toml'

Length of output: 383

rust/bin/agent/Cargo.toml (3)

25-25: Upgrade of prost from 0.12.4 to 0.13.1.

Upgrading prost to version 0.13.1 likely includes enhancements or bug fixes. It's important to review the release notes of prost to understand the changes and ensure they are compatible with the project. Additionally, testing should be conducted to verify that the upgrade does not affect existing functionality.


27-27: Update of tokio from 1.37.0 to 1.39.3.

The update to tokio version 1.39.3 includes performance improvements or new features. This change should be reviewed to ensure it aligns with the project's asynchronous runtime needs. It's also important to verify that this update integrates well with other asynchronous dependencies and does not introduce any breaking changes.


29-29: Upgrade of tonic from 0.11.0 to 0.12.1.

Upgrading tonic to version 0.12.1 likely reflects changes in gRPC support or related functionalities. This upgrade should be reviewed to ensure it is compatible with the project's gRPC implementations and does not introduce any regressions. Testing and verification are crucial to confirm that the new version integrates smoothly.

rust/libs/algorithm/Cargo.toml Show resolved Hide resolved
rust/libs/algorithms/faiss/src/lib.rs Show resolved Hide resolved
Signed-off-by: Kosuke Morimoto <ksk@vdaas.org>
Copy link
Contributor

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

Review details

Configuration used: CodeRabbit UI
Review profile: CHILL

Commits

Files that changed from the base of the PR and between f2d4fdf and 66a1753.

Files ignored due to path filters (1)
  • rust/Cargo.lock is excluded by !**/*.lock
Files selected for processing (1)
  • rust/Cargo.toml (1 hunks)
Files skipped from review as they are similar to previous changes (1)
  • rust/Cargo.toml

…rust/modify-directory-structure

Signed-off-by: Kosuke Morimoto <ksk@vdaas.org>
@kmrmt kmrmt merged commit 4111b89 into release/v1.7 Sep 10, 2024
27 of 29 checks passed
@kmrmt kmrmt deleted the backport/release/v1.7/feature/agent/rust/modify-directory-structure branch September 10, 2024 05:28
@coderabbitai coderabbitai bot mentioned this pull request Oct 10, 2024
2 tasks
@kpango kpango mentioned this pull request Oct 11, 2024
@coderabbitai coderabbitai bot mentioned this pull request Nov 19, 2024
2 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants